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ABOUT THE EDUCATION AND EMPLOYMENT RESEARCH CENTER 


Rutgers’ Education and Employment Research Center (EERC) is housed within the School of 
Management and Labor Relations. EERC conducts research and evaluation on programs and 
policies at the intersection of education and employment. Our work strives to improve policy 
and practice so that institutions may provide educational programs and pathways that ensure 
individuals obtain the education needed for success in the workplace, and employers have a 
skilled workforce to meet their human resource needs. For more information on our mission 
and current research, visit smlr.rutgers.edu/eerc. 


ABOUT RUTGERS’ SCHOOL OF MANAGEMENT AND LABOR RELATIONS 


Rutgers’ School of Management and Labor Relations (SMLR) is the leading source of expertise 
on the world of work, building effective and sustainable organizations, and the changing 
employment relationship. The school is comprised of two departments—one focused on all 
aspects of strategic human resource management and the other dedicated to the social science 
specialties related to labor studies and employment relations. In addition, SMLR provides many 
continuing education and certificate programs taught by world-class researchers and expert 
practitioners. 


SMLR was originally established by an act of the New Jersey legislature in 1947 as the Institute 
of Management and Labor Relations. Like its counterparts created in other large industrial 
states at the same time, the Institute was chartered to promote new forms of labor-management 
cooperation following the industrial unrest that occurred at the end of World War II. It officially 
became a school at the flagship campus of the State University of New Jersey in New 
Brunswick/Piscataway in 1994. For more information, visit smlr.rutgers.edu. 
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INTRODUCTION 


For students in community colleges, coursework in developmental (also called remedial) math 
can present a significant hurdle impacting both persistence and degree completion (Complete 
College America, 2012). These negative impacts of developmental math placement are 
disproportionately experienced by poor students and students color, exacerbating performance 
gaps (Attewell, Lavin, Domina, & Levey, 2006). Thus, remedial policies that were designed to 
increase college success have had the opposite effect. Most researchers and policymakers agree 
that some mathematics coursework may be necessary in college, but not all students need the 
same mathematics preparation for their careers. More broadly, mathematics sequences 
proceeding through algebra with the presumed goal of calculus—the pathway emphasized in 
K-12 and postsecondary education-seem at odds with workers’ reported mathematics use at 
work (Douglas & Attewell, 2017). Even researchers within the mathematics discipline recognize 
that college algebra often marks the end of formal mathematics study for students rather than 
serves as a prerequisite for further study (Gordon, 2008; National Research Council, 2013). 


Recognizing the practical implications of long sequences of developmental courses students 
often fail to complete, policymakers across the country have started modifying developmental 
coursework requirements (Bailey, et al., 2016). Corequisite remediation has emerged as a 
promising approach to reducing attrition rates related to developmental placement. Corequisite 
remediation allows students who would have been required to take noncredit developmental 
courses to proceed directly to credit-bearing college-level work. After the state passed its AB 
705 Law, California’s Community College system eliminated its developmental pathways and 
began offering credit-bearing courses to all students (Xia, 2017; Zinshteyn, 2019). Colorado has 
reformed developmental pathways in both English and mathematics, moving toward a 
corequisite model for the latter (McKay, Michael, & Khudododov, 2016). Some campuses in the 
City University of New York have experimented with corequisite placement into statistics, 
showing important evidence of effectiveness (Logue, Watanabe-Rose, & Douglas, 2016; Logue, 
Douglas, & Watanabe-Rose, 2019). 


In 2016, Union County College began an experiment with corequisite developmental 
mathematics as part of a grant from the U.S. Department of Education’s First in the World 
(FITW) program." In this experiment, students assessed as needing to take developmental 
mathematics courses would be eligible to receive a waiver from their developmental 
requirements and instead proceed to college-level mathematics courses. Students selected to 
receive a waiver would also be required to participate weekly in tutoring services offered by the 
college. The Education and Employment Research Center at Rutgers University served as the 
external evaluator for the study. The evaluation focuses on three key outcomes: continuous 
enrollment, passing college-level mathematics, and degree completion. 


1 Union County College had conducted a pilot study of the corequisite intervention in the academic year prior the 
grant. 


METHODS 


The evaluation used a randomized controlled trial (RCT) to assess program impacts on student- 
level outcomes in three domains — progress in college, academic achievement, and degree 
completion. RCTs are considered the gold standard in assessing the impact of educational 
interventions. The evaluation was designed to meet What Works Clearinghouse (WWC) group 
design standards without reservations. Those design standards inform the presentation that 
follows. 


Random Assignment and Sample Formation 


First-time students at Union County College assessed as needing developmental coursework in 
mathematics were eligible to participate in the RCT. Random assignment took place at the 
Union County College’s testing center. Students who placed into college-level math were not 
informed about the study. Students who placed into developmental math were informed about 
and invited to take part in the study. Those who agreed to join the study were asked to sign a 
consent form. Prior to each recruitment period, the evaluation team created a random 
assignment spreadsheet using a random sequence generator and used this list to fill numbered 
envelopes.” Consenting students were each given one envelope that contained their group 
assignment. The testing center staff would then record each student’s identifying information, 
envelope number, and group assignment. This procedure allowed the evaluation team to audit 
the random assignment process. 


Students were randomized into Union County College’s AMP-UP experiment each semester 
beginning in Fall 2016 until Spring 2019. The analysis presented here focuses on three-year 
outcomes for students randomly assigned in Fall 2016 and Fall 2017. 


Intervention Condition 


Students randomly assigned to the intervention condition were given a pre-signed registration 
form from their developmental math requirement that allowed them to register in college-level 
math courses. They were also required to visit at Union County College’s Academic Learning 
Center, a peer tutoring center, during their first math course. There were no specialized college- 
level-math sections for AMP-UP treatment group students, and math instructors were not told 
whether there were any AMP-UP students in their class sections. AMP-UP staff reminded 
students to go for tutoring. Students received weekly email reminders from AMP-UP staff to 
attend tutoring, prompts if they did not attend, and notes of praise if they did. Students in the 
intervention group also had access to enhanced content using Cengage MindTap software. 


Comparison Condition 


Students randomly assigned to the comparison condition were not given a pre-signed 
registration form and proceeded with their developmental course placement. Students in the 


2 The random sequence generator is available at https://www.random.org/sequences/ 
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comparison group also had access to Union County College’s Academic Learning Center, a 
resource available to all students, but they received no special reminders or prompts about 
using this resource from AMP-UP staff. However, if they went to the ALC they received the 
same emails of praise that the treatment group did. Regardless of group assignment, students 
were not required to immediately enroll in a math course. In some cases, students were allowed 
to enroll with their assignment, in later semesters. This was a decision made by college 
administration. 


Analytic Strategy and Data Sources 


Given the nature of the RCT, the evaluation uses an Intent-to-Treat design. We compare 
students as assigned and allow any post-random-assignment behavior to vary naturally. Thus, 
the RCT compares the treatment group — those who received waivers and were asked to 
participate in tutoring — with the control group — those who proceeded with the college’s 
business-as-usual policies. 


AMP-UP program staff provided the evaluation team with all of the data needed to conduct 
this analysis including student data, random assignment status, and student characteristics, 
along with term and course-unit data for all consenting students. 


Dependent Variables 


This report focuses on three outcomes across three dimensions of postsecondary student 
success. Progress in college is operationalized as the number of Fall or Spring terms enrolled 
during the three-year study period. Academic achievement is operationalized as whether a 
student ever completed a college-level course in mathematics in the three-year period. Degree 
completion is operationalized as whether a student completed a degree program in the three- 
year period. 


Independent Variables 


The main independent variable here is the binary indicator of whether a student was assigned 
to the intervention or comparison condition. In addition, we also collect measures of key 
student traits — gender, race/ethnicity, whether the student received a Pell grant in the random 
assignment term, and their score on the Accuplacer math exam prior to random assignment. 
Gender is a binary variable — male or female — and race/ethnicity uses the IPEDS classification. 
Pell grant status and Accuplacer scores are measures of student socioeconomic status and prior 
academic achievement, respectively. The What Works Clearinghouse defines SES and prior 
academic achievement as key measures of baseline equivalence for studies of postsecondary 
support interventions (What Works Clearinghouse, 2019). As such, they play a particularly 
important role in the analysis that follows. 


FINDINGS 


Table 1: Descriptive Statistics for the Corequisite AMP-UP Intervention and Business-As- 
Usual Comparison Groups, Fall 2016 and 2017 cohorts combined. 


Fall 2016 424 (50.8%) 460 (54.5%) 
Fall 2017 410 (49.2%) 384 (45.2%) 
Mean (SD) 26.4 (7.8) 26.1 (7.1) 
Male 385 (46.2%) 421 (49.9%) 
Female 448 (53.8%) 423 (50.1%) 
White 94 (11.3%) 104 (12.3%) 
Asian 17 (2.0%) 13 (1.5%) 
Black/African American 305 (36.6%) 319 (37.8%) 
Hispanic 292 (35.0%) 295 (35.0%) 
Other/Multiple Selected 30 (3.6%) 30 (3.6%) 
Unknown 96 (11.5%) 83 (9.8%) 
No 521 (62.5%) 478 (56.6%) 
Yes 313 (37.5%) 366 (43.4%) 


Mean (SD) 38.6 (13.9) 38.6 (13.8) 
*p<.05 


Table 1 presents the characteristics of students randomly assigned to the intervention and 
comparison groups in Union County College’s AMP-UP experiment. The data indicate that the 
random assignment process produced generally balanced groups. There are no statistically 
significant differences in the following average group traits: age, gender, race/ethnicity, and 
academic preparation as indicated by Accuplacer exam scores. The one exception to the 
observed group balance is financial aid status. A larger proportion of students in the 
intervention group received Pell grants in the semester in which they entered college. Though 
this difference is statistically significant, it is still within the WWC boundaries for baseline 
equivalence.* As such, we include both Pell status and Accuplacer in the models estimating 
treatment effects. 


3 The threshold for baseline equivalence is mean differences between 0.05 and 0.25 of the pooled standard deviation 
for each measure (What Works Clearinghouse, 2020). 


Table 2: Three-Year Outcomes for the Corequisite AMP-UP Intervention and Business-As- 
Usual Comparison Groups, Fall 2016 and 2017 cohorts — Unadjusted Comparisons 


Terms Enrolled — Mean (SD) 2.41 (2.08) 2.54 (2.03) 

Passed Any College-Level Math*** 229 (27.5%) 328 (38.9%) 

Earned a Degree 114 (13.7%) 125 (14.8%) 
*<.001 


Table 2 presents unadjusted group comparisons of three-year outcomes in three domains: 
retention, academic performance, and college completion. Students in the intervention group 
enrolled, on average, in a slightly higher — but not statistically significant - number of terms 
(2.5) than students in the comparison group (2.4). When we look at academic performance, we 
see that intervention group students were far more likely to complete a college-level math 
course (38.9%) than their peers in the comparison group (27.5%). This difference in college math 
completion is statistically significant. However, we do not observe that this advantage translates 
into degree completion, as students in the intervention group (14.8%) are only slightly more 
likely to have completed a degree within three years than students in the comparison group 
(13.7%) — a difference that lacks statistical significance. 


Table 3: Linear and Logistic Regressions Predicting Three-Year Outcomes 


Corequisite Intervention 0.06 1.68*** 1.06 
Group Member (0.09) (0.18) (0.15) 
Received Pell in Cohort Year 1.37%" 2.05*** 1.90*** 
(0.10) (0.23) (0.27) 
Accuplacer Score 0.02*** 1.04"** 1.03"** 
(0.003) (0.004) (0.005) 
>’ Odds Ratios Reported 
*< 001 


Table 3 adjusts the comparisons based on students’ academic preparation and socioeconomic 
status as indicated by Accuplacer scores and Pell recipient status, respectively. Adding these 
covariates does not substantially alter the estimates of the effect of treatment on the outcomes. 
We note that both the SES and prior academic achievement indicators are significant positive 
predictors of the three outcomes, which warrants their inclusion in the estimation of treatment 
effects. In Tables Al and A2, included as an appendix to this report, we report the results of 
further examination of these differences using propensity score matching and by using an 
expanded set of student characteristics. These tests yielded similar results. 


DISCUSSION 


Union County College experimented with corequisite developmental math as an alternative to 
the traditional model of mathematics placement testing and noncredit developmental 
coursework. In general, this approach involved allowing students assessed as needing 
developmental math coursework an opportunity to proceed directly to college-level 
mathematics with additional encouragement to utilize the college’s Academic Learning Center 
for tutoring support. Our formative evaluation showed that Union County College made many 
changes to its developmental education protocols over the course of the study period, which 
altered the experimental setting.* As such, the results of this summative assessment should be 
interpreted with caution and in that evolving context. 


The RCT was generally successful in producing balanced treatment and comparison groups; 
while the treatment group had a somewhat greater proportion of Pell grant recipients, this 
difference is within the WWC boundaries for baseline equivalence. The RCT had no attrition 
from the outcomes examined here. 


The outcomes assessment found that students assigned to the intervention group — those who 
had the immediate opportunity to proceed to college-level mathematics with support — 
benefitted primarily from the intervention itself. In other words, intervention group students 
were substantially more likely to have passed a college-level mathematics course within three 
years than their counterparts in the comparison group, who would have had to first complete a 
developmental mathematics sequence prior to enrolling in college-level math. Assignment to 
the treatment group did not, however, have a measurable impact on either student persistence 
at the college or on degree completion in the study period. This result remained consistent in 
the unadjusted comparisons in Table 2 and in the adjusted models in Table 3. 


The null findings for retention stand to reason; completing a single course is unlikely to 
determine a given students’ long-term postsecondary outcomes. The null findings with respect 
to degree completion may relate to factors immanent to and beyond the scope of the 
experiment. 


Union County College’s experiment follows a “randomized encouragement design” (RED) 
(West et al., 2008). In RED studies, participants assigned to the intervention group are provided 
with the means to take up the intervention and are encouraged to do so, but they are not 
compelled to comply with their assigned status. In this case, intervention group students were 
provided a pre-signed registration form that permitted them to enroll directly into college-level 
math. But neither intervention nor comparison group students were required to enroll in any 
math course at the time of random assignment. The encouragement was successful, however: 
At the end of three years, 74 percent of the intervention group had taken at least one math 
course, while only 65 percent of the comparison group had done so. But insofar as the 
intervention is only effective when students actually begin taking mathematics, which is a 
graduation requirement, the evidence generated by this experiment is limited by less-than-full 


4We document two such substantial changes below. For additional information from our formative analysis, see 
Coty, Edwards & McKay (forthcoming). 


uptake. In separate analyses (not shown) we estimated complier average causal effects; these 
did not vary substantially from the findings presented above. 


Union County College’s AMP-UP intervention existed alongside a rapidly changing 
developmental placement landscape at the college. In the first two years of the experiment — the 
cohorts evaluated in this report - developmental placement policies remained constant. 
Thereafter, there were a number of consequential changes that may have impacted students 
across cohorts. We briefly discuss two of these changes below. 


In Fall 2018, college-level math courses for liberal arts majors were opened to all Union County 
College students. Developmental math became effectively optional for most non-STEM, non- 
Health majors. These introductory college-level math courses were also redesigned to 
accommodate the changes to developmental placement. Any student, including control group 
students from the first two cohorts, was eligible to enroll in these redesigned courses. Students 
interested in majors that required college algebra were still subject to developmental placement 
based on Accuplacer scores. 


In Spring 2019, Union County College adopted the redesigned Accuplacer exam for placement 
decisions. This change most likely only affected new incoming students. In Fall 2019, college 
algebra was redesigned and became a dev-ed optional course. Similar to the Fall 2018 change, 
this shift may have affected students in the sample analyzed for this report. We speculate that 
the changes to Union County College’s developmental placement over time led to a 
convergence between the intervention and comparison conditions, which may explain the null 
findings for three-year student outcomes. 


As a comparison, another random-assignment study of corequisite mathematics, in which these 
contextual factors did not play a role, found that after three years, students placed in corequisite 
math earned more total credits and were substantially more likely to complete their degrees 
(Logue, Watanabe-Rose, & Douglas, 2016; Logue, Douglas, & Watanabe-Rose, 2019).5 The pace 
of change to Union County College’s developmental placement landscape may have limited the 
capacity of this evaluation to understand the broader impacts of this particular intervention. 


5 Those researchers also had access to degree completion data from the National Student Clearinghouse, which 
provided a more comprehensive estimate of degrees completed by participants. 
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APPENDIX 


Table A1. Odds of Intervention Group Assignment, Logistic Regression 


Yes 1.27 (0.13)* 1.30 (0.13)** 
Accuplacer Score 1.00 (0.004) 1.00 (0.004) 
on -- 1.00 (0.006) 

Female -- 0.84 (0.09) 

| RacelEtinnicity (ref White) 

Asian -- 0.62 (0.25) 

Black/African American -- 0.90 10.15) 

Hispanic -- 0.86 (0.14) 

Other/Multiple Selected -- 0.89 (0.26) 

Unknown -- 0.75 (0.26) 
N of Cases 1,677 1,675 

*p<.05 **p<.01 


Table A2. Propensity Score Matched Analysis of Student Outcomes 


Terms Enrolled — Mean (std. error) 0.008 (0.10) 


Passed any College-Level Math 0.11 (0.02)*** 


0.003 (0.02) 


PD <.01 
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